What Were They Thinking? Diagnostic Coding of Conceptual Errors in a Mathematics Learning Software Data Archive

Decades of research have demonstrated that students face critical conceptual challenges in learning mathematics. As new adaptive learning technologies become ubiquitous in education, they bring opportunities both to facilitate conceptual development in more focused ways and to gather data that may yield new insights into students’ learning processes. The present study analyzes data archives from a perceptual learning intervention designed to help students master key concepts related to linear measurement and fractions. Using algorithmic data coding on a database of 78,034 errors from a sample of sixth graders, both conceptual errors and other errors were captured and analyzed for change over time. Results indicate that conceptual errors decreased significantly. This approach suggests additional ways that such datasets can be exploited to better understand how the software impacts different students and how next generations of adaptive software may be designed to code and respond to common error patterns in real time.